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Focusing specifically on physics periodicals, I show that the journal Impact Factor is not correlated 
with Hirsch's /i-index. This implies that the Impact Factor is not a good measure of research 
quality or influence because the /i-index is a reflection of peer review, and thus a strong indicator 
of research quality. The impact gap between multidisciplinary journals and physics-only journals 
is significantly reduced when h is used instead of the Impact Factor. Additionally, the impact of 
journals specializing in review articles is inherently deflated using h because of the limited number 
of annual publications in such periodicals. Finally, a reordering of the top ranking journals occurs 
with h when only the physics articles of multidisciplinary journals are considered, falling more in 
line with the average physicist's interpretation of a journal's prestige. 



The journal Impact Factor (IF) has earned the 
adjective notorious for several reasonsi* 2 ^. First, in 
contradiction to healthy philosophies of science, the 
data used to determine the IF are not publicly avail- 
able. Second, authors have reported unethical editorial 
practices that have the motive of increasing a journal's 
IF, including editors insisting that additional references 
to their journal be added to a bibliography before 
accepting an article for publication. Third, publishing in 
high- IF journals is used as a filter for hiring tenure-track 
faculty and advancement to tenure. R. Monastersky's 
"The number that's devouring science" highlights other 
intriguing issuesi, and astutely notes that the pragmatic 
motto "publish or perish" has mutated to "publish in a 
high-impact journal or perish." 

The impact factor is, in spirit, a reasonable metric: 
the total number of citations divided by the total 
number of articles^. The three highest IF periodicals 
in which physicists publish were reported for 2003 as 
Nature (31.0), Science (29.2), and Reviews of Modern 
Physics (28. 2) 5 . Many physicists find it odd that 
Physical Review Letters, historically the most well 
respected physics journal, boasts a meager 7.0. Another 
indication that the IF rankings are questionable is that 
experts disagree with the rankings for their subfields: 
few of the top titles are recognized, and even fewer are 
considered prestigious. Upon inspection, most of the top 
IF journals in each subfield specialize in review articles. 
Review articles are typically highly cited, though only 
a few articles are published annually. This causes such 
journals' high (and arguably, anomalous) IF, while 
simultaneously making original research journals appear 
relatively weak. 

Based on the IF's several peculiarities, alternative 
measures of journal quality are being sought. In my 
opinion, the only reasonable way to rank physics journals 
is for physicists to measure the quality of published 
physics in each journal. While this may at first seem 
unrealistic, an indirect form of this type of peer review 
already exists in the form of citations. Articles of high 
quality or broad interest receive many more citations 
than articles of low quality or limited interest, and 
are therefore more scientifically influential. Extending 



this idea to periodicals, one can inspect the citation 
and publication history of a journal in a scientific and 
unbiased way to determine the average impact of its 
publications. I thus propose the /i-index as a logical 
measure of a journal's influence on science. 

The /i-index, developed by J.E.Hirsch to quantify 
the scientific research output of an individual, has 
become popular because it is logically sound, simple to 
understand, and, most importantly, simple to calculate 
with easily obtained data. An individual's /i-index 
is determined by searching a scientific database like 
Thompson's ISI Web of Knowledge 7 for all articles 
by an author, and ranking the output articles by the 
number of citations such that article 1 has the most 
citations; h is the rank of the lowest ranking article 
whose number of citations is bounded below by its rank. 
One can arbitrarily extend this procedure to journals, 
departments, institutions, or even zip codes. A graphical 
definition of the /i-index is shown in the inset of Fig. ^ 
If c(p) is the number of citations for paper p in the 
ordered list, then h is the intersection of c(p) with the 
line c'(p) = p. I have empirically observed that c(p) is 
bounded below by a right isosceles triangle with legs of 
length 2h, such as ABC; violations are more likely to 
exist for scientifically young subjects due to inadequate 
statistics. An (approximately) equivalent definition of h 
is the coordinate of the intersection of the hypotenuse 
BC with d{p) =p. 

The total number of citations can be used to develop 
a related, but arguably more poignant index. Consider 
ADE, a similar triangle of ABC, whose area (a) equals 
J2 P c (p)- Using the geometric definition, a new index, 
which I cannot resist naming the ?i-index, is the coor- 
dinate of the intersection of the hypotenus e DE with 
the line c'(p) = p, or mathematically, h = yf (a/2). The 
?i-index is a more comprehensive measure of the overall 
structure of c(p) for two reasons. First, h incorporates 
the most highly cited articles, while h basically ignores 
all articles with citations much greater than h. Second, 
h takes into account the body of articles with moderate 
numbers of citations, while h again ignores all such 
articles. 
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FIG. 1: (Color online) fa-index for fifty journals from major 
subfields of physics. (Inset) Graphical representation of the 
indices h and h as defined in the text. 



To illustrate the main difference between h and h, 
consider two similarly agedii, fully tenured, condensed 
matter physicists from the University of California, San 
Diego: L. J. Sham and I. K. Schuller. These professors are 
both well respected in this field, and have each received 
numerous awards for their scientific achievements. Both 
have h = 54, indicating that they have contributed 
approximately the same amount to science. However, h 
does not take into account that Sham's article with W. 
KohniS that introduced density functional theory— for 
which Kohn was awarded the Nobel Prize in 1998— is the 
most highly cited paper in the history of Physical Review. 
Professor Schuller, on the other hand, has a large body 
of work and is among the most highly cited researchers 
in physics, though his single most highly cited paper has 
an order of magnitude fewer citations than Sham's (and 
neither he nor his immediate coworkers has yet to win 
the Nobel Prize). I find that Sham has fi = 102, and 
Schuller has H = 76. Thus the Sham- Schuller Paradox 
is resolved using Ti. Degeneracies with the fi,-index will 
of course exist, but these will not be as qualitatively 
displeasing as in this and similar cases. 

For some of the individuals noted in Hirsch's 
original article, I find the following for the indices 
(h,h): E.Witten (112, 163), P.W.Anderson (96, 
164), S.Weinberg (89, 139), J.N.Bahcall (77, 102), 
D.J. Scalapino (76, 95), S.G.Louie (76, 97), R. Jackiw 
(69, 106), C.Vafa (67, 82), D.J. Gross (67, 106), and 
S.W. Hawking (62, 98). 

Applying the /i-index to individuals has proven very 



effective. Hirsch unequivocally showed that a large 
h, and more importantly d/i/dt, indicates a successful 
scientist. Additionally, a recent hind-sight study showed 
a strong correlation between h and committee peer 
review: individuals that were granted prestigious post- 
doctoral fellowships- in biomedicine by a committee of 
well known scientists in that field from 1990 to 1995 
had on average higher h- indices than other applicants 9 . 
Upon reflection, this study indicates the obvious: h 
measures how one's contributions are viewed by one's 
peers. Highly valued articles will receive many citations 
from the scientific community, which will result in a 
higher /i-index for the authors or publishing journal. 
Similar correlation of h with peer review was recently 
demonstrated in chemistryifi. 

I compared the rankings of scientific periodicals 
frequently targeted by physicists from a variety of 
subfields using h and h~ to evaluate the IF as a measure 
of research quality. The data used to calculate both 
indices were obtained using ISI to search by "source" , 
limited to 1990-2006, and the following document types: 
Article, Letter, Review, Correction, Editorial Material, 
or Note. The latter three were included because com- 
ments, errata, and retractions are listed under these 
headings. Journals that began publishing after 1990 
were excluded^. I chose sixteen years as an estimate of 
the time between entering graduate school and receiving 
tenure at a university. Additionally, this long timescale 
reduces the influence of high frequency fluctuations, 
such as those due to spectacular claims that are often 
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TABLE I: h, h, 2003 Impact Factor, total number of publi- 
cations, and the percent of uncited articles as of late March, 
2006 for the subset of a journal's publications noted in the 
text. "Based on title. 6 Based on automated filtesi&. 
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disproved soon after their initial publication (the IF 
only uses two years*). The data were harvested in a 10 
day window in late March, 2006; results are presented in 
Tabled 

Figure ^ shows the h-mdex of fifty physics-related 
periodicals. As I discuss below, the relatively large h 
of the top four is due to the fact that these periodicals 
publish articles from other scientific disciplines, which 
inflates their indices relative to physics-only journals. 
Physical Review Letters also has a jump relative to the 
remaining physics journals. This is probably because it 
publishes work from all of physics, while the others are 
for specific subfields. 

Figure [5] shows the journals ranked using h~, h, 
and the 2003 IF as published in the Journal Citation 
Report for twenty titles with h > 100. The arrows 
show how the individual journals change rank when 
analyzed with the different indices. A journal's rank 
changes by an average of 1.3 positions between Ti and 
h, 4.4 between Ti and IF, and 3.4 between h and IF. 
The ZF-based rankings are inconsistent with h and H. 
This means the IF is inconsistent with peer review, and 
is direct evidence that the IF is not a good measure of 
the quality of physics published in a particular journal. 
Interestingly, the greatest leap of —17 places was made 
by Reviews of Modern Physics. This may indicate that 
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FIG. 2: (Color online) h, h, and the 2003 IF for twenty 
physics journals with h > 100. The arrows show the re- 
ordering between the indices. 



h intrinsically solves the impact inflation problem of 
review journals. 

The large discrepancy between indices for Nature and 
Science relative to physics-only journals is due to the 
fact that their indices are inflated by work from other 
scientific fields; 14 '. Evidence of this inflation includes: 1) 
I can only identify ~ 8% of the top h articles in Nature 
and Science as physics research based on a qualitative 
assessment of their content by their titles*^. 2) Physics 
articles represent only a small fraction of the total 
number of articles in these periodicals, which means 
fewer physics articles per unit time. The total number 
of articles is important in determining h and h. Fewer 
articles means fewer citations, and thus an inherently 
lower /i-index (as seen with Reviews of Modern Physics). 
Together, these points indicate that the impact of 
multidisciplinary journals, regardless of the metric, is 
heavily weighted by topics other than physics. 

An important question naturally presents itself: how 
does the subset of physics articles in multidisciplinary 
periodicals compare with physics-only journals? To 
investigate this, an automated filter was used to select 
"physics" articles from the aforementioned data set of 
Nature publications. The selection criteria were based 
solely on the references of each article. An article 
initially qualified as physics if it contained at least one 
physics reference. The percentage of an article's total 
references that qualified as physics references served as a 
final, tunable filter. A reference was deemed a "physics 
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FIG. 3: (Color online) h (lower, circles) and fi (upper, 
squares) indices for Nature using the percent of physics ref- 
erences in individual publications as a filter. The highlighted 
data were used for the linear fit, which yields h p h y s = 285 and 
hphys = 499 when extrapolated to the ordinate- intercept. 



reference" if it contained generic search strings (astron, 
biophys, etc.) or specific abbreviations (phys fluids, 
j phys b, etc.), and did not contain other strings 
(physio, rehab, etc.)l&. The filter excluded nature 
and science because a disproportionate number of 
references contain at least one of these strings, which 
would have undermined this analysis. This method is 
certainly not perfect, but it is robust in the sense that 
perturbing the search strings does not significantly alter 
the results; the indices change by less than five percent 
when phys rev is purposefully excluded with the filter. 

Figure shows h and Ti for the qualifying Nature 
articles as a function of the final filter percentage. Each 
point on these curves was obtained by 1) selecting all ar- 
ticles whose bibliographies contained at least n% physics 
references, then 2) determining h and H for this subset 
of articles. The tail below fifteen percent is primarily 
due to physics references in non-physics articles. Both 
indices decay with increasing filter strictness as a result 



of the decreasing number of qualifying articles. The 
data around the inflection points (20-50%) were fit to 
lines, and the ordinate-intercepts were used to define 
physics-specific indices. I thus determined Ti p h ys — 499, 
and h p hys = 285. The latter is in good agreement with 
the manual analysis based on article titles: h p h ys = 263 
(hp h y 8 = 260 for Science). With h = 869 and h = 305, 
Physical Review Letters ranks higher than Nature (and 
presumably Science) for both indices. The origin of the 
large discrepancy between h p h ys and h for Nature and 
Physical Review Letters, respectively, is probably due to 
the large percentage of uncited articles in Nature (see 
Tabled). 

In summary, the journal IF was shown to be in- 
consistent with the /i-index. Insofar as h reflects peer 
review, and peer review reflects research quality, these 
results indicate that the IF is a poor measure of research 
quality. An additional benefit of h is its intrinsic 
grounding of the impact of review journals, whose IF is 
anomalously enhanced by their few annual publications. 
I showed that the impact of multidisciplinary journals is 
enhanced relative to physics-only journals by the more 
numerous non-physics articles contained therein. An 
analysis of the physics subset of Nature and Science 
revealed that Physical Review Letters has a greater h 
than either of these article subsets. This is in accord 
with Physical Review Letters' historical status as the 
most reputable physics journal. Based on these many 
observations, I conclude that the indices h and h are 
superior to the journal IF as indications of the quality 
of research published in a journal. 
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